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Abstract 



We study the asymptotic behaviour of the quantum representations 
of the modular group in the large level limit. We prove that each el- 

Cd . ement of the modular group acts as a Fourier integral operator. This 

provides a link between the classical and quantum Chern-Simons the- 
ories for the torus. From this result we deduce the known asymptotic 
expansion of the Witten-Reshetikhin-Turaev invariants of the torus 

^ , bundles with hyperbolic monodromy. 

i> : 

■^ , Quantum Chern-Simons theory was introduced twenty years ago by Wit- 

^T ' ten [^] and Reshetikhin-Turaev ||8[. It provides among other things invari- 

ants of three-dimensional manifold and representation of the mapping class 
group of surfaces, cf. ||l|, H for an exposition of the theory and [|| for a 

^S I survey on recent developments. This theory has a semi-classical limit, where 

the level, an integral parameter denoted by k, plays the role of the inverse 
of the Planck constant. In this paper, we are concerned with the torus and 

k>( \ its mapping class group, S1(2,Z). We study the large k behaviour of the 

^ ■ quantum representation of the modular group. 



The quantum representations may be equivalently defined with algebraic 
or geometrical methods. Geometrically, we consider a line bundle, called the 
Chern-Simon bundle, over the moduli space of flat G-principal bundles on 
the torus. Here G is a compact Lie group that we assume to be simple and 
simply connected. Then the modular group acts linearly on the space of 
holomorphic sections of the k-th. tensor power of the Chern-Simons bundle. 
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A part of this construction is standard in geometric quantization: to 
any compact Kahler manifold with an integral fundamental form one asso- 
ciates the space of holomorphic sections of a prequantum bundle. In this 
general context the usual tools of microfocal analysis have been introduced. 
In particular given a prequantum bundle automorphism, we define a class 
of operators similar to the Fourier integral operators Q, Q quantizing it. 



Our main result, theorem 9.1, says that each element of S1(2,Z) acts as a 
Fourier integral operator on the quantum spaces, the underlying action on 
the Chern-Simons bundle being defined through gauge theory. This estab- 
lishes a clear link between the quantum and classical Chern-Simons theories. 
As a corollary, we can estimate the character of the quantum repre- 
sentations of the hyperbolic elements of S1(2,Z). More generally, under a 
transversality assumption, one proves that the trace of a Fourier integral 
operators has an asymptotic expansion, which generalizes in some sense the 
Lefschetz fixed point formula y]- The characters of the quantum representa- 
tion of the modular group are the three-dimensional invariants of the torus 
bundles. In this way we recover the asymptotic expansion proved by Jeffrey 
1^, whose leading term is given in terms of the Chern-Simons invariants 
and the torsion of some flat bundles over the torus bundle. The proof in P] 
is completely different and relies on the reciprocity formula for Gauss sum. 
Our result is slightly more general since we treat any hyperbolic element 
with any simple simply connected group G. But, what is more important, 
we hope that our analytic method will work in other cases. In the compan- 
ion paper @, we prove similar result for the mapping class group in genus 

Besides the semiclassical results, we also give a careful construction of the 
quantum representations, comparing the geometric and algebraic methods. 
Strictly speaking, we do not have representations of the modular group but 
only projective representations which lift to genuine representations of the 
appropriate extension of S1(2,Z). The extensions appearing naturally are 
not the same in the geometric and the algebraic approach. 

The paper is organised as follows. In section |l|, we state our result about 
the asymptotic expansion of the trace of the quantum representations. In 
section ^, we introduce the phase space of the Chern-Simons theory for 
the torus, its symplectic structure and prequantum bundle. The relation 
with gauge theory is the content of section ^. In section ^ we introduce a 
complex structures on the phase space and the associated quantum Hilbert 
spaces. We exhibit basis in terms of theta functions. The modular group 
acts naturally on the previous datas but does not preserve the complex 
structure. In section |6| we identify the quantum spaces associated to the 



various complex structures. This leads to the definition of the quantum 
representations. In section |7| we compare these representations with the 
ones defined by algebraic methods. The next two sections are devoted to 
semi-classical results: section ^ on the identification of the quantum spaces 
and section ^ on the quantum representations. In a first appendix we prove 
basic facts on theta functions. In a second appendix we list some notations 
used in the paper. 

1 Characters of the quantum representations 

Let G be a compact simple and simply connected Lie group. The phase 
space of the Chern-Simons theory for an oriented surface S and group G is 
the moduli space of flat G-principal bundles over E. For a torus, this moduli 
space identifies with the quotient T /W , where T is a maximal torus of G 
and W is the Weyl group acting diagonally. The modular group S1(2,Z), 
being the mapping class group of the torus, acts on this moduli space. More 
explicitly, since T = t/A, with t the Lie algebra of T and A the integral 
lattice, we have a bijection T'^ /W ~ i^/(A^ xi W). Identify t^ with M? ® t, 
then an element A G Sl(2, Z) acts on the moduli space by sending the class 
of X to the class of {A (^ idt).x. 

Applying geometric quantization, we obtain a family of projective repre- 
sentations of the modular group indexed by a positive integer k. Since the 
construction is rather long, we only give in this introduction the represen- 
tation of the generators 

of the modular group. Let B be the basic inner product of the Lie algebra of 
G. We choose a set of positive roots and denote by 21 C i the corresponding 
open fundamental Weyl alcove. We identify the weight lattice A* C t* with 
a lattice of t via the basic inner product. The k-ih representation has a 
particular basis indexed by the set 2lnA;~^A*. For any A, /i G 2ln A;~"'^A*, let 

txfi = 6x^fj,exp{iTrkB{X,X)) 

and 

s^^ = iPk-^ Vori(t/A) Y^ (-l)^W eM-'^i^kB{X,w{fi))) 

wew 

where n is the rank of G and p is the integral part of n/2. 



If the rank of G is even, the map sending S and T to the matrices 
s\^ and t\^ extends to a unitary representation R^^ of the modular group. 
If the rank of G is odd, we obtain a representation R^^^ of an extension 
Mp(2, Z) of the modular group by Z/2Z. Recall that the metaplectic group 
Mp(2,M) is the connected two-cover of S1(2,R). Then Mp(2,Z) is defined 
as the subgroup of the metaplectic group consisting of the elements which 
project onto the modular group. The representation -Rq^j of the elements 
projecting onto S and T is given by the matrices ±sa^ and ^t\^. 

Theorem 1.1. Assume the rank of G is even, then for any hyperbolic ele- 
ment A € S1(2,Z), we have 

( _-\\2)Lp ik9{A®w,x) 

w&W,xeT^/ ' ^ ^' 

^w).x=x 



where 

• e = if the trace of A is bigger than 2 and e = 1 otherwise. 

• e{A(S>w,x) = TT{B{fi, p) - B{j, q) + B{-f,fi)) i/ x G T^ ~ (t/A)^ is the 
class of (p, q) G t^ and (7, fi) = {A® w){p, q) — {p, q). 

If the rank of G is odd, let A S Mp(2,]R) projecting onto an hyperbolic 
element A of the modular group. Then the same result holds for the trace 
of R^^^{A) except that the equivalent has to be multiplied by exp(i^ ind(j4)), 
where ind(A) G Z modulo 4Z. 

It is a general property of topological quantum field theories that the 
trace of the quantum representation of an element A of the modular group 
is the invariant of the mapping torus 

Ma := ((mVz2) X R)/{Ay,t) ~ {y,t + l). 

Here we ignore the the complications due to the framing of 3-dimensional 
manifold and the related fact that we only have a projective representation. 
For any (xi,X2) G T^ and w & W such that {A (S> w).{xi.,X2) = (xi,X2), 
consider the flat G-principal bundle Pa — >■ Ma whose holonomies along 
the paths 7(5) = [s,0, 0] , [0,s,0] and [0,0, s] are respectively xi, X2 and 
w~^. Then (27r)~^6{A w,x) is the Chern-Simons invariant of Pa- This 
is in agreement with the formula obtained by Witten using the Feynman 
path integral (heuristic) definition of the three-dimensional invariants. We 
refer the reader to Jeffrey's paper Q for more details. In particular the 
factor I det(id —Ai^w)\~^ appears as an integral of the torsion of the adjoint 
bundles over the moduli space of flat G-principal bundle over Ma- 



2 Lie group notations 

Let be a compact simple Lie algebra, and G the corresponding compact 
connected and simply-connected Lie group. Choose a maximal torus T of 
G and denote by t its Lie algebra. The integral lattice A of t is defined as 
the kernel of the exponential map t — )■ T. Since G is simply-connected, A is 
the lattice of i generated by the coroots a^ for the (real) roots a. 

Let the basic inner product B be the unique invariant inner product on 
such that for each long root a, B{a^,a^) = 2. Through the paper, we 
will use B to identify I with t*. The basic inner product has the important 
property that it restricts to an integer-valued Z-bilinear form on A which 
takes even values on the diagonal. 

We fix a set A-|- of positive roots and let t+ be the corresponding pos- 
itive open Weyl chamber. Let oq be the highest root and 21 be the open 
fundamental Weyl alcove 

21 := {A G t+/ ao(A) < 1} 

We denote by W the Weyl group of (G,T). Let £ : W ^ {±1} be the 
alternating character of W. 

3 The symplectic data 

In this section we endow T^ with a symplectic form uj and a prequantum 
bundle L, that is a complex Hermitian line bundle together with a connection 
of curvature -^uj. Furthermore we introduce commuting actions of the Weyl 
group and the modular group on L. 

3.1 A prequantum bundle on t^ 

Denote by p and q the projections t^ ^ i on the first and second factor 
respectively. Let w be the symplectic form on i^ given by 

oj = 2TTB{dp, dq). 

Consider the trivial complex line bundle L^2 over t^ with fiber C and con- 
nection 

d+-{B{p,dq)-B{q,dp)). 

Its curvature is jw, so it is a prequantum bundle. 



3.2 Heisenberg group and reduction to T^ 

Introduce the (reduced) Heisenberg group t^ x U(l) with multiphcation 

{x,u).{y,v) = {x + y,uvexjp{^u;{x,y))) 

The same formula defines an action of the Heisenberg group on L^2 = t^ x C. 
This action preserves the trivial metric and the connection. The lattice A^ 
embeds into the Heisenberg group 

A2 ^i^ X U(l), {p,q) -^ {p,q,eMi^B{p,q))) 

Using that B takes integral values on A, we prove that this map is a group 
morphism. Hence we get an action of A^ on L^;2 by automorphisms of pre- 
quantum bundle. 

By quotienting, we obtain a symplectic form on T = i^/A^ with a 
prequantum bundle L := t^ x C/A^ over T^. 

3.3 Weyl group 

Consider the diagonal action of the Weyl group W on i^ and lift this action 
trivially on the bundle Li2 . Since W acts on t by isometries, W acts on t^ 
by linear symplectomorphisms and on L^2 by isomorphisms of prequantum 
bundle. 

The Weyl group preserves the integral lattice A. Consider the semi-direct 
product H^ X A^ where W acts diagonally on A^ . Is is easily checked that the 
actions of A^ and W on the prequantum bundle over t^ generate an action of 
W X A^. Then, quotienting by A^, we obtain an action of I^ = (W x !\?)/h? 
on L. Since the action of the Weyl group on the base T is not free, we will 
not consider the orbifold quotient T /W and its prequantum bundle. 

3.4 Modular group 

Let us consider the symplectic action of the modular group F = Sl(2, Z) on 
t^ given by 

A.{p,q) = {ap + hq,cp + dq), ^^( ^ 

The trivial lift to the prequantum bundle Lj2 preserves the metric and the 
connection. Furthermore this action together with the action of A^ define 
an action of the semi-direct product F x A^. To prove this, one has to 



use that B{p,q) is integral when p,q £ A and even if furthermore p = q- 
Consequently we get an action of the modular group on L by prequantum 
bundle isomorphisms. Observe that the Weyl group action on L commute 
with the modular action. 

4 C hern- Simons theory 

We explain how the definitions of the previous section can be deduced from 
gauge theory. Our aim is only to motivate the constructions. No proof in 
the paper relies on the gauge theoretic considerations. 

The phase space of the Chern-Simons theory for an oriented surface S 
is the moduli space of representations of the fundamental group of S in G. 
When S is a torus, the fundamental group is the free Abelian group with two 
generators, so each representation is given by a pair of commuting elements 
of G unique up to conjugation. In the same way that G/ Ad G ~ T^/W, 
one shows that these representations are conjugate to a representation in 
the maximal torus T, uniquely up to the action of the Weyl group. So the 
moduli space of representation for the torus is T'^/W. 

4.1 Gauge theory presentation 

Consider the space ri^(S, g) of connections of the trivial G-principal bundle 
with base S. It is a symplectic vector space with symplectic product given 
by 

n{a,b) = 27r / B{a,b). 

The gauge group C°°(S,G) acts on il-'^(S,0) by symplectic affine isomor- 
phisms: 

g.a = Adg a — g*9 

where 6 € Q,^{G,q) is the right-invariant Maurer-Cartan form. Each gauge 
class of flat connections is determined by its holonomy representation. The 
quotient of the space of flat connections by the gauge group may be viewed 
as a symplectic quotient which defines a symplectic structure on the moduli 
space of representations. 

In the case S is a torus, we can avoid this infinite dimensional quotient 
proceeding as follows. Represent S as the quotient M^/Z^ with coordinates 
x,y. Then the map 

i^^n^(T.,Q), {p,q) ^pdx + qdy 



is a symplectic embedding where the symplectic product of t^ is the one 
of section |3.1| . This embedding is equivariant with respect to the action of 
VF XI A'^ on t^ and the morphism from VF x A^ to the gauge group sending 
an element w £ W to the constant gauge transform w and {p, q) € A^ to 
exp{—{xp + yq)). Furthermore each gauge class of flat connections intersects 
the image of the embedding. 

4.2 Prequantum bundle 

Consider the trivial line bundle with base il^(S,g) and connection d -\- ia, 
where a is the primitive of fi given by 

a\aib) = ^n{a,b). 

This is a prequantum bundle and the gauge group actions lifts to it in such 
a way that it preserves the trivial metric and the connection. Explicitly, the 
action is given by 



g.[a,u) 



(g.a,eyip(—2i7:W{g)—iiT B{g*6,a)]u 



where 9 € i^^{G,Q) is the left invariant Maurer-Cartan one- form and W{g) 
is the Wess-Zumino-Witten term 



Wig) = [ ~g*X 

JM 



Here M is any three-dimensional compact oriented manifold with boundary 
S, ^ G C°°{M^ G) any extension of g and x is the Cartan three-form defined 
in terms of the left or right-invariant Maurer Cartan forms by 

Since B is the basic inner product, the cohomology class of x is integral. 
Assume now that E is a torus and consider the equivariant embedding 



of t^ in il^(S,0) defined in section 4T. By pulling back, we obtain a pre^ 



quantum bundle on t^ together with an action oi h? yiW on it. It is not 
difficult to check that this bundle and this action are exactly the ones we 
introduced in section 0. 



4.3 Mapping class group 

The group of orientation preserving diffeomorphisms of S acts symplecti- 
cally on $7^(5], 3). The trivial lift to the prequantum bundle preserves the 
connection and the trivial metric. After quotienting by the gauge group, 
this defines an action of the mapping class group on the moduli space of 
representation and its prequantum bundle. 



When S is a torus, we recover the action of T introduced in section 3.4. 
For any ^ G T, define the diffeomorphism of the torus 

(Pa{x, y) = {dx - cy, -bx + ay) 

where a, 6, c, and d are the coefficients of A. On one hand, we recover the 
usual formula by considering the basis a = (0,-1) and f3 = (1,0). Indeed 
ipA{oi) = aa-\-hj3 and ^a{P) = ca + dj3. On the other hand, 

ip\{pdx + qdy) = {ap + bq)dx + {cp + dq)dy 

which corresponds to the action of section ^^. 



5 Quantization 

Let us begin with a brief description of the general set-up. Consider a sym- 
plectic manifold (M, u) with a prequantum bundle L — > M. Assume (M, w) 
is endowed with a compatible positive complex structure, so a; is a (1, 1) form 
and —iuj{Z,Z) > for any non vanishing tangent vector Z of type (1,0). 
Then the prequantum bundle has a unique holomorphic structure such that 
the local holomorphic sections satisfy the Cauchy-Riemann equations: 

V^s = 0, for any vector field Z of type (1, 0). 

The quantum space associated to these data is the space H^{M,L^) of 
holomorphic sections of L . It has a natural scalar product obtained by 
integrating the punctual scalar product of sections against the Liouville 
measure |a;"|/n!. 

In a first subsection we introduce complex structures on T and define 
a basis of the quantum space by using theta functions. We also describe 
the action of the matrices S and T of F in these basis. These results are 
standard. We provide proofs in appendix. Next we we move on to a subspace 
of equivariant sections with respect to the Weyl group action, the so called 
alternating sections. We compute the actions of S and T in this space. 



5.1 Complex structure and theta functions 

Denote by T-Lj^ the Poincare upper half-plane 

7^+ = {x + iy/x, y e M, y > 0}. 

Let r € ?^+. Identify t^ with ic = t^C by the isomorphism sending (p, q) to 
p + rg. Hence i^ becomes a complex vector space and T inherits a complex 
structure. One may compute the symplectic form uj in terms of the complex 
coordinate C, = p + Tq 

uj = B{dC,dC). 

T — T 

It is a positive real form of type (1,1)- So the prequantum bundle L has a 
unique holomorphic structure compatible with the connection. We denote 
by H^{T ,L) its space of holomorphic sections. 
Consider the section of L^2 

s = exp(i7r5(C,g)) 

Its covariant derivative is 2iirB{dC, q)CS)s. Since this form is of type (1, 0), s is 
holomorphic. Furthermore s doesn't vanish anywhere. So the holomorphic 
sections of L^ identify with the sections over i^ of the form fs^ such that 
/ : i^ — > C is holomorphic and fs is A^-invariant. 

As previously, we embed A* in t via the identification given by the basic 
inner product. Recall that A C A*. For any /j, € k~^A*, consider the theta 
function 

e^,fc(p,g)= Yl exp(2i7r/c(§5(7,7)-i?(C,7))) 

This series converges uniformly on compact sets to a holomorphic function, 
it depends only on /i mod A. 

Theorem 5.1. For any integer k, the sections Q^^k-s^j where /U runs over 
k^^A* mod A, are A"^ -invariant and form an orthonormal basis of H^{T'^ ,L^). 
Furthermore, 



'27r\"/2/ 2i7r \"/2_ 
.~k) \T-f) 

where Vol(t/A) is the Riemannian volume determined by B. 



1^ fc,i2 /27r\"/2/ 2ot \"/2 
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Recall that the modular group acts on T and its prequantum bun- 
dle. The induced action on the sections of L^ doesn't preserve the space 
H^{T ,L), because of the complex structure. Actually, A £ T acts as a 
holomorphic map from (T ,jV) to (T ,JAt) with 

ar — b 

At 



-CT + d 
So for any r, A acts as an isomorphism 

One may compute explicitly this isomorphism in the basis of theta functions 
when A is the matrix S or T. We make explicit the dependence in r in our 
notations to avoid any ambiguity. 

Theorem 5.2. For any r G 71^ and fi £ k^^A*, one has 

s. (e;;,fc4) =c Y. exp(-2i7rfci?(^, ^0)65:^ si, 

fi'ek^'-A* inodA 

with C = (5.T/i)"/^A:-"/2 Vol(t/A)-i and 

T.{Ql^,s';) = eMi^kBif,,i,))ell 4., 

Here (r/i)"' ^ is the determination continuous with respect to r and equal 
to 1 when t = i. 

5.2 Alternating sections 

The action of the Weyl group on T is holomorphic with respect to the 
complex structure defined by any r G 71+. Let us consider the alternating 
sections of L^, i.e. the sections ^ satisfying 

For any fi € k~^A*, let 

wew 
Recall that we denote by 2t the fundamental open Weyl alcove. 

Theorem 5.3. The family (x^,fc, ;^ € 2t fl k^^A*) is a basis of the space of 
alternating holomorphic sections of L . 

11 



Proof. Using that the Weyl group action preserves A and B, we check that 
for any w G W, 

So Xfi,k is alternating. 

For any root a and integer n, the orthogonal reflexion with respect to 
the hyperplane a~^{n) belongs to the affine Weyl group VF xi A. So for any 
/i G a~^{n), there exists w £ W with £{w) = 1 such that w{^) = // modulo 
A. Hence x^.fc = -Xw{ti),k = -X,i,k, so X/.,fc vanishes. 

Recall that the affine Weyl group VF xi A acts simply transitively on the 
set of components of t\ UQ,^„a~^(n) and that 21 is one of these components. 
The result follows from theorem ^.1[ D 

The modular action and the action of the Weyl group on T and L 
commute. So the representation of the modular group preserves the subspace 
of alternating sections. 

Theorem 5.4. For any r G 7^+ and /i G 2t n k^'^K* , one has 

S.{xlA) =C Y. (-l)'^"^exp(-2z^A;i?(M,«;(^'))x^Cfe 4. 
/i'G2lnfc-iA*, 

with C defined as in theorem \5.^ and 

T.{xlk4) = eMiT^kB{fi,fi))xl-l 4.r 



Proof. The second formula follows from theorem p.2| using that the Weyl 



group acts isometrically on t. Let us prove the first one. By theorem 5.2 



S-ixlA) =C E (-1)'^"^ exp(-2z7rfci?(u;(/.), ^0) O^^s^r 



/i'Gfc~iA* mod A, 



-C E (-1)'^"^ exp{-2^nkB{^^,^,')) e%,^^,s%,. 



/i'efc^^A* mod A, 



--C Yl exp(-2i7rfc5(/x,^'))x5rfc4r 

/i'Gfc-iA* modA 

--C Yl exp(-2i7r/cB(/i,w;(M'))x^(>),fc4.r 



/i'eanfc-iA*, 

wGW 
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In the last line, we used that the affine Weyl group acts simply transitively 
on the set of connected components of t \ |J a^^{n) and that x^ vanishes if 
fi G a~^{n). Finally, 

^'e2lnfc-iA* 
since the X/i,fc's are alternating. D 

6 Geometric quantum representation 

We introduce a representation of the modular group on the quantum spaces. 
To do this we identify the various spaces H^{T'^ , L^) via the sections 0^ ^sj:. 
Unfortunately the norm of these sections and the action of the modular 
group depend on r as it appears in theorems ^ and ^. We introduce 
half- form bundle to correct this. 

6.1 Half- form bundles 

Let us begin with some definitions. Consider a symplectic manifold M with 
a prequantum bundle L and a positive compatible complex structure. Then 
a half-form bundle is a complex line bundle 5 over M with an isomorphism 
from 5"^ to the canonical bundle of M. A half-form bundle admits a natural 
metric and a natural holomorphic structure making the isomorphism with 
the canonical bundle a morphism of Hermitian holomorphic bundle. The 
quantization of M with metaplectic correction is then the space of holo- 
morphic sections of L tensored with 5. The scalar product is defined by 
integrating the punctual norm of sections against the Liouville measure. 

Let us return to our particular situation. Consider f2 G A""!^ such that 
for a basis (7j) of A, one has 

fl(7i A . . . A 7„) = 1. 



il is uniquely defined up to a plus or minus sign. For any r G "H^, we defined 
e , 

T2^ic/(A + TA), [p^q]^[p + rq]. 



a complex structure jV on T via the isomorphism 



So the holomorphic tangent bundle of (T , jV) is naturally isomorphic to the 
trivial bundle with fiber tc- Consequently the canonical bundle is naturally 
isomorphic to the trivial bundle with fiber A^iJ. Denote by ^r the section 
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of the canonical bundle which is sent into the constant section equal to 
by this trivialization. 

Lemme 6.1. The section $7,- is holomorphic and has a constant punctual 
norm equal to (^^)" Vol(t/A)^^. 

Proof. Introduce an orthonormal basis (ui) of t and denote by (pi) and (qi) 
the associated linear coordinates of i^. Let CI be the complex coordinate 
Pi + Tqi. Then if (7j) is a basis of A, one has 

dCr A . . . A dC(7i, • • • , 7n) =dpi A ... A dpn{-fi, • • • , 7n) 

= ± Vol(i/A) 

where the plus or minus sign depends on the orientation of the various basis. 
Consequently 

Jl, = ± Vol(i/A)-idCr A . . . A dC" (1) 

Now, by definition the Hermitian product of two tangent vectors X, Y of 
type (1,0) is given by jOj{X,Y). Since 

a; = 27r ^ dpi A % = i ^ d^. A dCl 

the vectors Saj are mutually orthogonal and 

In |2 2z7r 

' ^^ ' T - T 

So the dQ are mutually orthogonal and 
which implies 

K;A...A<f = (^)" 

and concludes the proof. D 

Let 6 he a complex vector line and (p be an isomorphism 5®^ -^ A*°Pt^. 
Let Jl^/^ € She such that 
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For any r G T-Lj^^, the trivial bundle 5r with base T and fiber 5 is a half- 
form bundle, with the squaring map sending Q}'"^ into Q.r- By the previous 
lemma, the constant section equal to 0^" is a holomorphic section of 5r 
with constant punctual norm equal to (^^)" Vol(t/A)~^/^. 

Instead of sections of L^, consider now sections of L^ ® 6r- The multi- 
plication by $7^' ^ is an isomorphism 



of vector space. We deduce from theorem ^.11 and lemma 3.1 the 
Theorem 6.2. For any t G %+, the sections 

Ql^kS^^ ^J^/^ ^l € k-^h* mod A, 

form a basis of H^iT ,L^ 6r)- They are mutually orthogonal and 

For any ti and T2 in V.^, let ^^^^^^.fc be the isomorphism from i7^^ (T^, L'^^ 
6r,) to H^^{T^, L^ 5r^) defined by ' 

By the previous theorem, ^^^^^^.A; is a unitary map. 

6.2 Modular action 

Let j4 € r be the matrix with coefficients a, 6, c and d. The map ipAip, q) = 
{ap + bq,cp + dq) is a holomorphic map from (T^,jV) to (T^,jA.r)- Using 
the coordinates introduced in the proof of lemma B?^, we show that 

^\VLA.r = {-CT + dyVLr. (2) 

We will lift the action of A to the half-form bundles in such a way that it 
squares to (y?^)~^. Since {—cr + d)'^ doesn't admit a preferred square root, 
we have to pass to an extension of the modular group. 

Let r2 be the set of pairs [A, e) where A (^ T and e is a continuous 
function from 71+ to C satisfying 

e(rf = i-CT + dr if ^=(^ ^ 
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r2 is a group with the product given by {A,e).{A' ,e') = {AA',e") where 
e"(r) = e(^V)e'(r). r2 acts on the product n+ x T"^ x 5 

{A,e).{T,p,q,zn^/^) = ( "^~ ap + bq,cp + dq,zeiT)n^/^ 

\—CT + a 

Restricting to a particular value r, we obtain a morphism from 5r to 5a.t 
lifting the action of ^4 on T . The important point is that the square of this 
isomorphism is the natural map between the canonical bundles of (T ,jV) 
and (T ,jat)- Since the Hermitian and holomorphic structures of a half- 
form bundle are determined by the corresponding canonical bundle, the 
morphism 6r — > 5a.t that we consider is a unitary holomorphic isomorphism. 
As previously we lift trivially the action of T to the prequantum bundle. 
Then we obtain for any (A, e, t) &T2 x Ti^ an isomorphism 

V>lA,e)-^,r ■■ H%T\L''®5r)^Hl,{T\L''®5A.r) 

By trivial reason, it is a unitary map. Miraculously, these isomorphisms all 
fit together. 

Proposition 6.3. For any ti,T2 G Ti^ and any {A,e) € T2, the diagram 



A.T^,A.T2 






v, 



commute. 



Proof. It is sufficient to prove it for {A,e) = (T, 1), {S,e) or (id, — 1) since 
these elements generate F. The actions of (T, 1) and {S, e) in the basis 
SJ^ j^s^Q^''^ are given by the same formulas as in theorem |5.2| except that 
the constant C has to be replaced by 

C' = Ceir) = e(i)/t-"/2 Vol(t/A)-i 

This proves the result because C" does not depend on r. D 

Corollary 6.4. For any t, the map 

is a unitary representation ofT2 on H^(T ,L ®6r). 
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Consider now the diagonal action of the Weyl group on T . Its trivial 
lift to the half- form bundle acts holomorphically and preserves the metric. 
So we have a unitary representation of W on H^{T'^,L^ 6r)- It is easy 
to see that this representation commutes with the one of T2- This gives a 
representation of r2 on the subspace of alternating sections that we denote 



by i?2 . By theorem |5.3| , we have 



Theorem 6.5. The coefficients of the matrix ofRf^{S,e) and Rf^{T,l) in 
the basis Xii,k ® f^^' ^, /i G 21 n k^^A* are respectively 



-4^1-— ^(-l)^(-)exp(-2.vrA:B(/x, 



w 



Km'))) 



and 



5f,y exp{i7rkB{fj,, ^)). 



7 Algebraic projective representation 

Consider the category of representations of the quantum group Uq{Q) for 
q being the root of unity. This category has a subquotient, called the fu- 
sion category, which is a modular tensor category. Following Q, we can 
define a natural projective representation of the mapping class group of any 
2-dimensional surface with marked points on appropriate spaces of mor- 
phisms in this category. In particular, for the torus, we get a projective 
representation of the modular group that we define in this section. 

7.1 The representation Roo 

Denote by p the half sum of positive roots and by /i^ = 1 -|- p(ao) ^^^ 
dual Coxeter number. Let k be an integer bigger than /i^. Then the set of 
admissible weights at level fc — /i^ is 

Ck := A* n (fc - /i^)2l 

For any X, fi ^ Ck, let 



and 



A* 



kA 



-1/2,,^,, ^^ , _^,,, f 2iTT 



lA+l ^ (_i).H exp -^i?(^(A + p), ^ + p) 



wGW 



(111 
—B{\\ + 2p) 
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Consider the action of the modular group on the real projective line 
induced by the standard action on M?. Choose a base point Lq € P^(M). 
Let Too be the set of pairs {A, 7) where A G T and 7 is a homotopy class 
(with fixed endpoint) of a path in P^(R) from A.Lq to Lq- T^o is an extension 
by Z of the modular group, the product being given by 

(A7)-(^',7') = (^^',^(7')*7). 

Assume that Lq = [1,0]. Then Too is generated by s = {S, [cp]), t = (T, [1,0]) 
and 7 = (id, [7]) where (p and 7 are the paths 

(p{t) = [sin(t-),cos(i-)], 7(t) = [cos(t7r),sin(t7r)], 

with ^ t s^ 1. 

Theorem 7.1. Too admits a representation Roo on C-"* determined by 

Roo{s) = (exp(-zf c)sa^), Roo{i) = (ixfi), Rooil) = {exp{i^c)6xf,). 

The reader is referred to the book of Bakalov-Kirillov |j|] for a detailed 
exposition on modular tensor category and the fusion category of Uq{Q). 
The formulas for s\fj, and txfj_ are given in theorem 3.3.20, that they give 
a projective representation is the content of chapter 3.1, cf. remark 3.1.9. 
The definition of the central extension is in chapter 5.7, the case of the torus 
being treated in example 5.7.7. Note also there is a misprint in formula for 
sx^, compare with proposition 3.8 of [^]. 

7.2 Comparison of the representations Rf^ and R^ 

To compare i?!'* with R^o , we introduce a third extension r4 of the modular 
group. By definition, r4 consists of the pairs {A, e) where A &T and e is a 
continuous function from H^ to C satisfying e(r)^ = (— cr + d)^", with a, b, 
c, d the coefficients of A. The product is given by the same formula as for 
r2. Let Z4 = {ibl,ibz}. The morphism 

r2 X Z4 -> r4, {A,e,u) ^ {A,eu) 

is onto with kernel {(id, 1, 1), (id, —1, —1)}. Thus we have a representation 
i?4 of r4 given by 

R^(A, en) = uRf{A, e), {A, e) G La, n G Z4. 

Recall that Too is generated by s, i and 7. 
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Lemme 7.2. There exists a morphism ^ from Too onto r4 such that 
^i>{s) = {S,e) with e{i) = e~'> 

Proof. Consider the groups rj^ and F^ defined exactly as Too and r4 except 
that we replace the modular group by F := S1(2,M). We will construct a 
morphism from F^ onto F^ whose restriction to T^o is ^ ■ Let j be the 
morphism from the unit circle U C C into F 

. , . / reel u — im n 
y imu reelti 

Since F admits a deformation retract onto i(U), one has a bijective corre- 
spondence between the morphisms from F|^ to F^ covering the identity of 
F'* and the morphisms from J*F|^ to J*F^ covering the identity of U. On 
one hand 

j*F^ ~ {(n, e) € U xC*/ e^ = u^^™} =: U^ 

because for t = i, —ct + d = u~^ if a, b, c, and d are the coefficients of j{u). 
On the other hand 

j*rl^{iu,9)/ expiie)=u^}=:V^ 

where we send {u,6) into (j(u), [7]) with 7 the path 

j{t) = [cos{^9),sm{^d)], tG[0,l]. 

In particular s and 7 are identified with {i, it) and (1, — 27r). The morphism 
we are looking for is 

Here jA+j is the number of positive roots of G. The images of s and 7 are 
given by a straightforward computation using that dimG = 2|A+| + n. D 

Lemme 7.3. For any k, there is a morphism C,k '■ Fqo -^ C* such that 
with Xk = h"^ dimG/k where /i^ is the dual Coxeter number of G. 
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Proof. The value of Xk does not matter for the proof. We only use that Too 
is generated hy s, t and 7 with the relations 

(7s ) = id, (st) = s , 7 and s are central 

as it is asserted in [Q], example 5.7.7. D 

So by the previous lemmas, we define a representation of Too by 

We will prove it is the same as Roo- To do this we have to identify C *= with 
H!^{T^,L^ (8> (5r)- We need the following well-known fact. 

Lemme 7.4. For any k, the map sending A into —^ is a bijection between 
Ck and 21 n k-^A* 

The identification is then defined by sending the canonical basis of C '^ 
into the basis x^,fc ® ^2^/^, /i € 21 fl A:"^A*. 

Theorem 7.5. The matrices of Roo{s), Rooit), Rooil) in the basis X{\+p)/k,k'< 
Q}'"^ , X G Ck are respectively given by 

(exp(-zfc)sA^), {ixf,), (exp(z|c)(5A^). 

where (sx^) and {tx^) are the matrices defined in section [j'. 



Not only does it prove that R^o = Roo, but it also proves theorem 7.1. 

Proof. This follows from theorem |6.5| . Since ^(s) = {S,e) with e(i) = e~*4 
fdimG^ 



and Ck{s) = exp(i^ dim G-r^), the matrix of Roo{s) is 



-,,Vo.(t/A) E(-l)«-'exp(-2..^B(^..(^)) 






wew 



because 



A* 



A 



Vol(t/A) _^. 



Vol(i/A*) 



Vol"(t/A). 
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To conclude it suffices to compare with the formula defining sa^- Since 
^(t) = (T, 1) and because of the value of Cki't)^ the matrix of Roo(t) is 

where we have used that 

; — = — TidimG — c) 

which follows from Freudenthal's strange formula. D 

8 Complex structure dependence in the semiclas- 
sical limit 

8.1 Fourier integral operators 

Let M be a symplectic compact manifold with a prequantization bundle 
L. Consider two pairs {ja,^a) and {jb,Sb) consisting of a positive complex 
structure of M together with a half-form bundle. We say that a section ip 
of Hom(5a, Sh) is a half-form bundle isomorphism if its square at x is given 
by 

where TTb,a,x }sthe projection from Eb,x ■= T^' (MJb) to Ea,x ■= T^' (MJa) 
with kernel Eb^x- 

For c = a,b, denote by Ti.k{c) the space of sections of L^®5c holomorphic 
with respect to jc- Consider a sequence {Sk) such that for every k, Sk is 
an operator Tikia) — > "Hkib). The scalar product of Tikia) gives us an 
isomorphism 

}iom{nk{a),nk{b)) c^Tikib) (g)nk{a). 

The latter space can be regarded as the space of holomorphic sections of 

{L''(g)6b)M{L''(^6a)^M^, 

where M^ is endowed with the complex structure {jb,—ja)- The section 
Skix,y) associated in this way to Sk is its Schwartz kernel. 

We say that {Sk) is a Fourier integral operator with symbol the half- form 
bundle isomorphism (p if 

Sk{x, y) = {^) ^E'^ix, y)f{x, y, k) + Oik-^ (3) 

where 
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• £■ is a section of LM L ^ M^ such that ||-E'(x, y)\\ < 1 if x ^ y, 

E{x, x) = u CS) u, \/u ^ Lx such that ||u|| = 1, 

and Be = modulo a section vanishing to any order along the diago- 
nal. 

• f{.,k) is a sequence of sections of 6i, ^ 6a — > M^ which admits an 
asymptotic expansion in the (J°° topology of the form 

whose coefficients satisfy dfi = modulo a section vanishing to any 
order along the diagonal. 

• The restriction to the diagonal of the leading coefficient /o is equal to 
if, if we identify 5b (8> 6a with Hom((5a, 5h) using the metric of 5a- 

8.2 The maps ^ri,T2,fc in the semi-classical limit 

Recall that for any r G "H^, we defined a complex structure on T together 
with a half- form bundle 5r- Consider the morphism fTi,T2 form 5^ to 5r2 
given by 

^T2 — Tl/ 

where the square root is determined in such a way that it depends continu- 
ously on Ti, T2 and equal to 1 when ri = T2. 

Lemme 8.1. The map v'ti,t2 is a half-form bundle isom,orphism,. 

Proof. Introduce an orthonormal basis (lij) of t and denote by {pi) and (gj) 
the associated linear coordinates of i^. Let Q. be the complex coordinate 
Pi + TQi. Let Et = span(9^i, . . . , 9^n) be the space of vector of type (1, 0) 
with respect to the complex structure jV- Let tTt2,ti be the projection from 
-Et-2 to E-j-^ with kernel -E't-2- One has 



which implies that 



\r2 — Ti- 
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and then 

^« A . . . A O = (^^)"dC4 A ... A C 



Finally recall that by equation (||) the squaring map of 6r sends Q^''^ into 
±Yol{i/A)d(^A...AdC!^. a 

Theorem 8.2. For any ti,T2 € 71+, the sequence (^ri,T2,fc)fc is a Fourier 
integral operator with symbol ipr^ ^t-2 • 



Section 8.3 is devoted to the proof. 



8.3 Proof of theorem |8.2 



As previously we lift everything from T^ to t^. Denote by p2, 92 (resp. pi,qi) 
the coordinates on the left (resp. right) factor of t^ x \?. Let Qi = Pi + Tiqi 
for i = 1,2. 

Let us first compute the section E and the leading coefficient /o of (^. 
Let L^_2 be the trivial line bundle over \? with the connection defined in sec- 
tion ^. Consider the bundle L^2 M L^p, endowed with the holomorphic struc- 
ture compatible with the connection and the complex structure (JV2 > ~Jti ) 
oft^xt^. 

Lemme 8.3. The section of L^2 ^ L^2 

I'TT 

^S(C2-Cl,C2-Cl 

T2 -Ti 

is holomorphic and restricts to the constant section equal to 1 on the diag- 
onal. The morphism 9?ri,T2 is sent to the constant section equal to 



•.T2 -Ti- 

by the isomorphism between Iioin{6ri , dri) and br^ ^br^ induced by the metric 

of (Jri • 

The first part is proved by a straightforward computation and the second 



part follows from lemma |6.l| . By theorem |6.2| , the Schwartz kernel of ^Ti,T2,fc 
lifts from T^ x T^ to t^ x t^ into 

Sk{P2,q2,Pi,qi) = Y. e;^,(p2,92)e;;,(pi,gi)(4oi/2) m {4,Ti'^') 

AtG(fc-iA*)/A 



E UniiP2,Q2,Pi,qi)(.s';^n'/') K (s^^fi' 



/ie(fc-iA*)/A 

7i,72e/i+A 
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where the coefficients are given by 

7-,,,^, =exp(22^A;(f 5(72,72) -i?(C2, 72) - ^i3(7i,7i) + i?(Ci,7i)))- 

Lemme 8.4. For any compact set K of i such that ivTn A = 0, there exists 
C > such that for all P2,Q2,Pi, Qi satisfying qi — q2 ^ K one has 

\Sk{P2,q2,Pi,qi)\ <.Ce-^/^ 

This shows that the sequence of Schwartz kernels of ^Ti,T2,fc is a 0{k^°°) 
outside the diagonal. For the proof of the lemma we need the following 
general estimates. 

Lemme 8.5. For any C > there exists C > such that 

7GZ" 

For any C > 0, for any compact K of M" and for any subset P of Z" such 
that Kr\{I7'\P) = 0, there exists C > such that 

7eZ"\p 



Proof of lemma 8.4 ■ By a straightforward computation we obtain that 

\f,,^,A, ^<\= exp(-A;vr(^^^|(?2 - I2? + ^^ki " 7i 
Hence for some positive C, 



If s'^ M-c^ I < p-fcC'(l72-g2|^+|(g2-gi)-(72-7i)P) 

So with q = q2 — qi, 

-fcC(|72-g2p+|(/-7|2) 



72e/j+A 

7GA 



71:72£a«+A 72G/J+A 

7GA 



7GA 7GA 



By lemma B.5, there exists C" > such that the first factor is bounded 



by CA;"", the second one by C'e ' , and these estimates are uniform 
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with respect to /i, qi and q2 such that qi — q2 & K. Since the cardinal of 
(A;^^A*)/A is /i;"|A*/A[, we obtain with a larger C that 

Ate(fc-iA*)/A 

7i,72 6a«+A 

which proves the result. D 



Using exactly the same method and lemma 8.5 with P = {0}, we show 
Lemme 8.6. There exists C > such that 

Yl |/72,7i(P2,'?2,Pl,'?l)4 ^^r,\ < Ce-'^/^ 

MG(fc-iA*)/A 
7i.72eAt+A, 71^72 

for all p2, q2, Pi and qi such that \qi — q2\ ^ R/2 with R = min||7|, 7 G 
A\{0}}. 



So up to a 0{k °°), Sk{p2,Q2,Pi,Qi) is given on a neighborhood of the 

7.7 
Lemme 8.7. We have 



diagonal by the sum of the /-y-y where 7 runs over k ^A*. 



» — ^ / ik \ "/S 

E hfiP2,q2,Pi,qi)=( =-) Vol(t/A) 

f-^ \T2 — T\J 

x^exp(^^i3(A + C2-Ci,A + C2-Ci) 
All ^^2-ri 

Proof. Introduce a basis (vtj) of A*. Let 7 € fc^-'^A*, write A:7 = X] XiTTi. One 
has 

/7,7fe, 92,^1, gi) = exp(^- (^^-^) (y) -6(^7, kj) - 2iTTB{C2 - Ci, M 
=m(x) exp(— i(?7, x)) 

where « is the complex valued function of R" given by 

u{x) = exp(-2 (^— ^) (-^) E-^(^^'^i)^^^J 

and 

r/i = 27r5(C2-Ci,7ri). 
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Let (7j) be the dual basis of (vtj). Then using that B{^i,^j) is the inverse 
of B{TTi,TTj) and that the determinant of (i?(7j,7j)) . . is the square of the 
volume of t/A, we prove that the Fourier transform of u is 

*(0 = i^y" Vo.(./A) exp(4 (^) (^) E B(-..vm^ . 

The Fourier transform of v{x) = u{x) exp{—i{ri, x)) is 

for real rj. This is also verified for any rj € C" by analytic prolongation. By 
Poisson's summation formula, 

Let us compute {)(27r,^). Since ^ 7?i7i = 27r(C2 — Ci)) we have 

Y, i?(7^, 7j)(2vr^i + ^*)(2<, + Vj) ={27:fB{X + C2 - Ci, A + C2 - Ci) 

where A = ^ S,iji G A. So ■i)(27r^) which is equal to u{2tt^ + r/) is given by 
^) Vol(t/A)exp( ^i3(A + C2-Ci,A + C2-Ci: 

T2 — Tl^ ^T2 — Tl 

which concludes the proof. D 

Lemme 8.8. There exists e > and C > such that 

E |exp(^^i?(A + C2-Ci,A + C2-Ci))4^^n ^Ce-'^/^ 

■'^ — ' I \T2 — Ti / 

AgA\{0} 

for all p2, q2, Pi and qi satisfying \qi — q2\ '^ ^ and |pi — P2I ^ £• 
Proof With a straightforward computation, we obtain that 



exp( ^|A + C2-CiP)s^2^^n 

\T2 — Ti / 



^ ^ g-(a|M|2+26i?(At,<?)+c|g|2) 



where q = q2 — qi A* = A + p2 — Pi and a, b and c are the real numbers 



ZTT 

^T7('r2 -r2 + ri -ri) 



|T2 -ril2 
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= 1 =-r^{T2Tl - T2T1) 

^7^ /I |2/ \ 1 |2/ \\ 

T2I (ti - ri) + In I (t2 - r2)) 



|t2 -TlP 



Write 

a|;u|2 + 265(//, q) + c|g|2 = a|/i + f gp + (c - ^)|g|2. 

Using that a is positive, one proves that there exists e > such that 

\q\ <eand|p2-pi[ ^ e ^ f |/i + ^gp + (c - ^)|(?|2 ^ 
for any non-vanishing A G A. So 



2 



AgA\{0} ^^ ^^ AeA\{0} 



when \q\ and |p2 — Pi| are smaller than e. We conclude with lemma 8^. D 
Collecting together the previous lemmas, we obtain 

2^) (:^^3-) Vol(t/A) 
X exp(-^^i?(C2 - Ci,C2 - Ci))(4^'^') ^ {-s';,'^'^')+RkiP2,q2,Puqi) 

\ T2 — Ti / 

where the remainder satisfies for some e > and C > 0, 

ki -Q2\,\Pi -P2K e ^ |-Rfc(P2, 92,^1, gi)| ^ Ce"^/'^. 



Using lemma B.5, this proves theorem K^ 



9 Asymptotic properties of the quantum represen- 
tations 

9.1 Definitions 

Let M be a symplectic compact manifold with a positive complex structure 
j, a prequantization bundle L and a half- form bundle 5. Consider a sym- 
plectomorphism ^ : M ^ M together with automorphisms ^l and ip of the 
bundles L and 6 respectively which lift $. We assume that ^l preserves the 
connection and metric of L. 
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Let l-Lk be the space of holomorphic sections of L (g)S. Consider a family 
(Sk) such that for every k, S^ is an operator T-Lk -^ 'Hk- The Schwartz kernel 
of Sk is a holomorphic section of 



where M^ is endowed with the complex structure (j, —j). We say that {Sk) 
is a Fourier integral operator associated to ^l with symbol ip if the Schwartz 
kernel sequence is of the form 

— ) F\x,y)g{x,y,k)+0{k-°-) 

where 

• F is a section oi LML ^ M^ such that ||F(x,2/)|| < 1 if x / ^{y), 

F{(^{x),x) = ^l{u) (8) n, Vn E L^ such that ||n|| = 1, 
and BF = modulo a section vanishing to any order along the graph 

• g{., k) is a sequence of sections of 6M6 — > M^ which admits an asymp- 
totic expansion in the C°° topology of the form 

gi; k) = go + k~^gi + k~^g2 + ... 

whose coefficients satisfy dgi = modulo a section vanishing to any 
order along the graph of $^^. 

• The restriction to the diagonal of the leading coefficient go is equal to 
ip, if we identify 6^6 with Hom((5, 6) using the metric of 6. 

Let us explain the relation with the Fourier integral operators of section 
^.1| . Let $(j) be the complex structure obtained by pushing forward j 
with $. Consider a half-form bundle 5' of the complex manifold (M, <I>(j)) 
together with an isomorphism ipi : 5 ^ 6' whose square is equal to 

Then the isomorphisms $i and ipi induce a linear isomorphism <I>* from 
Tik to the space T-L'f^ consisting of the sections of L^ (Si 6' holomorphic with 
respect to $(j). Now suppose that 

Sk = TkO<^,:nk^nk, A; = 1,2,... 
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for an endomorphism Tk : 7^^ — > Tik- Then comparing the definition of 
section |8.1| with the previous one, we prove that (T^) is a Fourier integral 
operator with symbol (p2 in the sense of section |8.1| if and only if (Sk) is a 
Fourier integral operator associated to ^l with symbol (p = (f2 ° fi- This 
applies to the representation R2 defined in corollary |6.4| . Indeed for any 
{A, e) € r2, R2{A, e) is the composition of a pull-back with the map ^At.t, 



which is a Fourier integral operator by theorem 8.2. 

Since the half-form bundle 5r is the trivial bundle, we can identify its 
automorphisms with functions on T^, the correspondence being given by 
(fxi^^'"^) = f{x)Q'^''^. We use this convention in the sequel for the symbols 
of the Fourier integral operators. 

Theorem 9.1. For any r and {A,e) G r2 the sequence 

R2{A,e) ■.H^{T'^,L^^5)^H^iT\L''^5), A: = 1,2,... 

is a Fourier integral operator associated to the prequantum lift of A to L. 
Its symbol is the constant function equal to 



a{A,e) 




Applying theorem 8.2 with ri = r2, the representation of the Weyl group 



W is also given by Fourier integral operators. 

Theorem 9.2. For any r and w ^W the sequence 

w:H^{T^,L^^6) ^ H^{T^,L'' (g) 6), A; = 1,2,... 

is a Fourier integral operator associated to the prequantum. lift of w to L. 
Its symbol is the constant function equal to 1. 

9.2 Metaplectic group 

Let 5 be a symplectic vector space with a positive compatible complex 
structure j. Denote by £■ = ker(id-l-ij) the space of vectors with type 
(1,0). Let Sp(S') be the symplectic group of S. Using the complex structure 
we introduce a group Mp(S', j), isomorphic to the metaplectic group of S. 
Mp(S', j) consists of the pairs {A, z) such that A G Sp(5) and z is a complex 
number satisfying 

z^ = dei{g'^'KE,9E : E ^ E). 
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Here T^E,gE is the projection from E onto gE with kernel E. The product 
of Mp(5, j) is determined by the condition that the projection onto the 
symplectic group is a group morphism and that the identity is the pair 
(id, 1). We shall also consider an extension Mp2(5', j) by Z4 = {ibl,ibi} 
of the symplectic group. It is defined as the set of pairs (^, z) such that 
z^ = dQ\?'{g~^'iTE,gE '■ E — > E). The product is determined by the condition 
that the map 

Mp(5,i)xZ4^Mp2(5,i) 

sending (A, z, u) into {A^ zu) is a group morphism. 

We apply these constructions to5' = t©i = M^(8'i with the complex 
structure given by some r in the upper half-plane. Then the symbols of the 
operators defining the representation R2 belongs to the metaplectic group. 

Proposition 9.3. We have a group morphism from, T2 to Mp(t © t, jV) 
sending {A,e) into {A(S>idi,a{A,e)) with 

a{A,e)=e{r){^^f' 

Proof. Observe that for any g £ Sp(S'), the endomorphism 

T^E,9E o ia'^y ■■ A*°P^* ^ A'°P{gEy -^ A*°P^* 

is the multiplication by det{g~^7rE^gE : E ^ E). 

Let us apply this to g = ^ (8) idt. By equation (^) and the condition 
e(r)^ = {—CT + d)*^, the pull-back by 5^^ is multiplication by e^(r). By 
lemma ^.1| , tt*^ „e is the multiplication by ( 4 )"• This implies that 
(^4 (g) idt, (y{A, e)) belongs to the metaplectic group Mp(t © t, jV). 

One shows that the map is a group morphism by extending it to the 
group defined as r2 by replacing Sl(2, Z) with Sl(2, R) and using a continuity 
argument. D 

For any A G S1(2,M), let d{A,T) = det{A-^TiE,AE : E ^ E) with E the 
complex polarization determined by the complex structure p + rq. Then in 
the proof of the previous proposition we showed that 

d{A,T) = (-CT + d)^I^. (4) 

We will use this equation several times in the sequel. Assume that the rank 
of G is even, so n = 2p. We have a morphism from T into r2 sending A into 
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{A, {—CT + d)P). Composed with the morphism provided by proposition 3.2, 
we obtain the group morphism 

r^Mp(tet,jV), A^{A(g)idi,d{A,Tf). (5) 

Assume now that the rank of G is odd, n = 2p + l. Introduce the subgroup 
Mp(Z,r) of Mp(M2,r) consisting of the pairs {A,z) G T x C* such that 
z'^ = d{A,T). Using again (Q), we prove that this group is isomorphic to r2, 
the isomorphism being given by 

-1/2 



{A,z) eMp{Z, t) ^ (a, z{-CT + d)P{^^)-'^'] GTa 



Finally composing this morphism with the one of proposition 9.3, we obtain 
the group morphism 

Mp(Z,T) ^Mp(tei,jV), {A,z)^{A®[di,zd{A,T)P). (6) 

Considering the representation of the Weyl group, we obtain a morphism 
into the extension Mp2(i© t, jV) of the symplectic group. 

Proposition 9.4. We have a group morphism from W to Mp2(i0 t, jV) 
sending w into (id]g2 (S)w, 1) 

More generally, if w is an element of the orthogonal group of t, then 
(id]R2 ^w, u) belongs to the metaplectic group (resp. the extension by Z4) if 
and only if u'^ = detu; (resp. n^ = 1). 

9.3 Index computation 

As previously consider the metaplectic group Mp{S,j) of a symplectic vec- 
tor space endowed with a complex structure. Let Mp^(S', j) be the subset 
consisting of the pairs {g, z) such that 1 is not an eigenvalue of g. Then we 
defined in Q an index map 

ind:Mp,(5,i) ^ Z/4Z 

It is continuous and takes distinct values on each of the four components of 
Mp^(S', j). To compute it, we only need the two following properties. If E 
has dimension 2, then 

ind{g,z) = k+\{l-{-lf+^) (7) 

where fc E Z is such that the argument of z belongs to [^/c, |(A; + 1)[ and e is 
equal to if the trace of g is bigger that 2 and to 1 otherwise. Furthermore if 
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(51, ^i) G Mp^(S'i,ji) and (52, ^^2) G Mp^(S'2,j2) then igi®g2,ziZ2) belongs 
toMp,(5ie52,jiei2) and 

ind(5'i 52,^:1^2) = ind{gi,zi) +ind(52,^2) (8) 

The elements {g,z) of Mp2(5', j) such that 1 is not an eigenvalue of g, also 
have an index defined modulo 4Z. It is such that 

ind{g, i z) = k + md{g, z) 

if {g,z) G Mp^ (S", j) . In the following we compute the index of some elements 
of the metaplectic group of 5 = t © t endowed with the complex structure 
determined by r G H^. 

Lemme 9.5. For any hyperbolic A G S1(2,M) and w in the orthogonal group 
of i, we have 

ind(j4 ®w,z)= md{A ® idt, z) 

where z is any complex number such that {A^idi, z) belongs to Mp2(t©t, jV)- 

Proof. A being hyperbolic, 1 is not an eigenvalue oi A® w. Since 

{A (g) idt, z).{\d^2 0w, n) = (j4 (g) w, zu) 

The fact that (^©idt, z) belongs to Mp2(t©l, jV) implies that {A®w, z) also 
belongs to Mp2(t©t, jV)- Let us prove that they have the same index. Since 
the index is locally constant, the result is straightforward if w belongs to the 
special orthogonal group. Otherwise we may assume that w is a reflexion 
and that A is the diagonal matrix with coefficient 2, 1/2. Let us decompose 
t as a direct sum of orthogonal lines. The complex structure jV preserves 
the associated decomposition of M? (g) i. So using (g) it is sufficient to prove 
that 

ind(yl,u) = ind(— ^,ti) 

One may assume that {A,u) G Mp(M^,T) so that {-A,iu) G Mp(M^,r). 
Then the result follows from formula (0). D 

We can give explicit formulas for the index of (^(gidt, z) by decomposing 
M^ © t into a direct sum of M^'s as we did in the previous proof. 

Lemme 9.6. If n = 2p, for any A G SU(2,R), we have 

md{A © idt, d{Af) = 2ep 
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where e is equal to if the trace of A is bigger than 2 and to 1 otherwise. If 
n = 2p+l, for any {A,z) G Mp^(M-^,r), we have 

m.d{A (g) idt, d{AY z) = 2ep + md(^, z) 

where e is defined as previously. 

In the second case, the index of {A,z) is given by (^). With these 
formulas we obtain the index of any element in the images of the morphisms 
(I) and (I). 

Proof. Working with the decomposition of M? ^ I, we only have to consider 
n = 2. We have 

ind(yl A, d{A)) = 2 ind{A, z) 

where z^ = d{A). We conclude with formula (0). D 

9.4 Trace estimates 

Under a transversality condition, the trace of a Fourier integral operator 
admits an asymptotic expansion and we can explicitly compute the leading 
term in terms of the symbol. Next theorem has been proved in Q. We 
restrict to the case of T to simplify the statement. 

Theorem 9.7. Let ^ be a symplectomorphism ofT whose graph intersects 
transversally the diagonal. Let ^l be a prequantum bundle isomorphism of 
L lifting $ and {Tf, : H'^{T'^,L^ (g) Sr) -^ H^T"^ , L^ ® 6r)) be a Fourier 
integral operator associated to ^l with symbol f . Then for any fixed point 
X of^, there exists a sequence (a^^x) of complex numbers such that for any 
N, 

tv{Tk) = Yl ^- K,o + a^,ik-' + ... + a,,Nk-^ + Oik-""-')) 

x/ ^(x)=x 

where for any x, u^ is the trace of ^l{x) : L^ ^ L^. Furthermore, if 
{Tx^,f{x)) is an element o/Mp2(t © t, jV); then 

■ind{T^<S>,f{x)) 

axfl 



det(id-r^$)|V2" 



We apply this to estimate the character of the representation i?!^*. First 
we have 

tT{RfiA,e)) = -^ V {-lY^^hT{w.R2iA,e)) 



\W\ ^^ 
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Then by theorems 3.1 and 9.2, ■w.R2{A,e) is a Fourier integral operator 
associated to the prequantum hft A (S) w. Its symbol is the constant map 
equal to a{A, e). By |9.5| , the index of {A w, <t{A, e)) doesn't depend on w. 
We compute easily the action of the prequantum lift of AiSi w at the fixed 
points and obtain the 

Theorem 9.8. For any (A, e) G r2 such that A is hyperbolic, we have 

^n{A,e) ike{A®w,x) 

mGT^/ {A®w).x=x 

where 

• n{A,e) is the index of {A (^idi, a {A, e)) 

• 9{A®w,x) = iT{B{n,p)—B{'y,q)+B{'y,fi)) ifx is the class of{p,q) S t^ 
and {j, n) = [A w){p, q) - {p, q) . 

We can explicitly compute the indices with lemma ^^. For the statement 
in the introduction we used the two morphisms (^) and (|6|). 



A Proofs of theorem |5.1| and |5.2 

A.l The basis of H%T^, L^) 

Recall that the holomorphic sections of L^ identify with the sections over t^ 
of the form fs such that / : t^ — ?> C is holomorphic and fs is A^-invariant. 
As shows a straightforward computation, this invariance is equivalent to 

f{p + p, q + q)= f{p, q) exp{-2iTTk{B{C, q) + ^B{q, q))) 

for all p, q in A. Then to prove that the sections &f_i^kS form a basis of the 
holomorphic sections of L^, we decompose the functions / as a Fourier series 
in the p variable with coefficients depending on q. Then the holomorphy 
and the equivariance in the g-directions translate into a condition on the 
coefficients, leading to the result. 

Let us shows that the sections 0^,fcs'^ are mutually orthogonal and com- 
pute their norms. Let m be the Riemannian volume of t. The Liouville 
measure |a;"'|/n! of t^ is equal to {2iT)'^m{p) ® m{q). The scalar product of 
fs^ and f s^ is given by 



(2vr)" / f{p,q)f'{p,q)\s{p,q)\^''m{p)®m{q) 
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Here D is the fundamental domain \^xiiii + . . .+Xn^J-n/ {xi) € [0, 1]"}, where 
(;Uj) is a basis of the lattice A. Now a straightforward computation shows 
that 

0/.,fc0/.',fc|s(p,g)P'' = ^ c^,y(g)exp(2mA;5(j»,7' -7)) 

7'eAi'+A 

where the diagonal coefficients are given by 

Cmil) = exp(i7rfc(r - f)B{q -J,q- 7)). 

Integrating with respect to p, one deduces that the scalar product of Qfj,,k 
and Q^',k vanishes when fJ- ^ fJ,' mod A. Furthermore, 

||G^,fcs'^||2 = (2^)"Vol(l/A) Yl j c^Mm{q) 

7eM+A •'^ 

Using that i is the disjoint union of the —7 + D when 7 runs over /U + A, we 
obtain 

V / c^,^{q)m{q) = / exp(i7rA;(r - f)B{q, q)) m{q) 
7eM+A-^^ -^t 

n/2 



\k(T 



■~k{T — t) 
which ends the computation of the norm. 

A. 2 The action of S and T in the basis of Theta functions 



Let us prove theorem ^.2| . Denote by 99^ the map sending (p, q) into {ap + 
bq, cp + dq). Recall that Ct = P + tQ and Sr = exp(i7ri3 ((",-, q)). Then 

V'aCa.t = —J, (/?^SA.r = exp(i7r — r)^-^ 

—CT + a V — cr + a / 

In particular, for A = T~^, this gives 

^AifiCA.r)s'A.r) = fiCM 
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so that 

VaKP\,,)= E exp(2mfc(i±li?(7,7)-i?(C.,7)))s^ 

Using that B take integral even values on the diagonal of A^ , one shows that 
exp(z/i:7ri?(7,7)) = exp(i/c7ri?(/u,/i)) for any 7 € /i + A. This implies that 

VA{QtpA.r)= Y. exp(i/c7r5(/x,^))e;;^;,s^ 

7GAt+A 

which proves the second formula of the theorem. 
Assume now that A = S~^, then 



Applying to the theta functions, we have 

^*KkAr)= E exp(-^i3(7 + Cr,7 + Cr))^^ 

7GAt+A 



Applying Poisson summation formula, we obtain after some computations 
^A. 



that y^* {Q^'k Sa.t) is equal to 



-) Vol(i/A)-^ E exp(2i7rA:(§i?(7,7)-^(M + Cr,7)))4 

Since k~^A* = \J{n' + A), where /x' runs over A:~^A* mod A, this is equal to 

-) Vol(t/A)-^ E exp(-2i7rA:i?(/.,M'))0;;',fc^' 

/^'gfc-iA* 
mod A 

which ends the proof. 

B Index 

Lie group notations: 

3, B Lie algebra of G and its basic inner product; 

T, t, A maximal torus, its Lie algebra and integral Lattice; ^ 

21, W open fundamental Weyl alcove and Weyl group; ^ 

£ : W ^ {il} alternating character; ^ 
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Moduli space and its quantization: 



P^Qj C = P + TQ projections from i^ onto t, complex coordinates; 3J., xl 

uj, L(2 symplectic form and prequantum bundle of l^; 3J 

L prequantum bundle of T^ ; |3.2| 

s, Qfj.,k section of L^2 and theta function; |5.1| 

{Xfi,k)ii basis of alternating sections of H^{T , L^); ^^ 

Modular group extensions and their representation 

r2, i?2 extension by Z2 of F, representation in H'^{T'^,L^); p^ 
i?|^* representation in the subspace of alternating sections; 6.2 

TcxD, Roo extension by Z of F and its representation; T^T 
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